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1 Image Sensors with Multiple Integration/Read Cycles 
2 

3 The present invention relates to solid state image 

4 sensors. More particularly^ the invention relates to 

5 CMOS-type image sensors in which multiple integration 

6 and read cycles are performed between resetting of 

7 the sensor pixels. This provides the basis for 

8 obtaining images which are low in noise and/or having 

9 a relatively wide dynamic range and/or which are 

10 resistant to flicker induced by mains frequency 

11 flickering of artificial light sources. 
12 

13 Solid state image sensors fabricated using CMOS 

14 technology provide a low cost imaging solution, as 

15 compared with CCD image sensors, for applications 

16 such as digital still cameras, camcorders, web 

17 cameras etc. Many kinds of CMOS imaging circuit 

18 architectures are possible, including three main 

19 types as illustrated in Fig. 1: (a) passive pixel 



1 type; (b) 3 -transistor active pixel type; and (c) 4- 

2 transistor active pixel type. 
3 

4 A disadvantage of types (a) and (b) above is what is 

5 known as "reset noise". This type of noise makes the 

6 reset potential of the photodiode forming part of 

7 each pixel circuit uncertain, and this uncertainty 

8 adds to pixel-to-pixel and frame -to -frame variable 

9 noise in both still and video images. 
10 

11 The rms magnitude of this reset noise is known to be: 
12 

Nrst = V(kT/C) volts 

14 

15 Where k is Boltzmann's constant, T is absolute 

16 temperature in Kelvin, and C is the capacitance of 

17 the node being reset. 
18 

19 Pixel type (c) above does not suffer from this type 

2 0 of effect if a special type of buried diode is 

21 employed which enables the reset node to be entirely 

22 depleted of free charge. Photodiode arrays of this 

23 type ("pinned" photodiode arrays) require at least 

24 one manufacturing process step in addition to the 

2 5 normal CMOS process, and incur an overhead cost of at 

26 least one additional transistor per pixel as compared 

27 with a 3 -transistor active pixel. 
28 

2 9 CMOS image sensors may also suffer a performance 
disadvantage in environments which are lit by 



30 



31 artificial light sources whose intensity varies 



# 



3 

rapidly in time with the AC mains supply frequency, 
or at some harmonic of this frequency. For example, 
fluorescent lights "flicker" at twice the supply the 
frequency. In these conditions the exposure mode of 
most un- shuttered CMOS sensors causes horizontal 
banding interference in the image, which may also be 
seen to scroll vertically. 

It is known to correct this flicker effect by making 
the exposure time substantially equal to one period 
of the flickering source, or an integer multiple 
thereof . 

A third disadvantage, which is common to most image 
sensors, is that the range of scene luminances that 
can be captured in one frame (i.e. the dynamic range 
of the sensor) is significantly limited. In 
practice, certain noise mechanisms limit the lowest 
luminance levels, and combinations of supply voltage 
and circuit design cause areas of the scene which are 
above a critical luminance to be clipped or 
saturated. Typically, the dynamic range available in 
one frame is limited to about 6 0 dB, but most real 
scenes contain luminance ranges greater than this. 

The present invention relates to image sensors having 
modes of operation in which the operation and timing 
of certain active pixel sensor arrays substantially 
eliminates the effects of reset and flicker noise, 
whilst also expanding the available instantaneous 
dynamic range. 
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1 

2 The invention may be applied to any active pixel 

3 architecture which supports a non-destructive read of 

4 pixel values; e.g. type (b) 3 -transistor types as 

5 discussed above, but not type (a) passive pixels. 

6 The invention exploits the fact that, with pixel' 

7 types which support non-destructive read operations, 

8 it becomes possible to perform multiple, staggered 

9 read operations without the pixels being re-set 

10 between read operations, so that the data read at any 

11 particular point in time represents the cumulative 

12 signal integrated up to that point since the last 

13 time the pixels were re-set. 
14 

15 The invention is particularly intended for use with 

16 CMOS type image sensors, image sensor systems and 

17 cameras. 
18 

19 In accordance with a first aspect of the invention, 

20 there is provided a method of operating a solid state 

21 image sensor having an image sensing array comprising 

22 a plurality of active pixels, the method comprising: 

23 resetting each said pixel; 

24 after a first predetermined period of time 

25 reading a first output from each said pixel so as to 

26 obtain a first set of image data having a first 
2 7 dynamic range; 

28 without resetting said pixels, after a second 

2 9 predetermined period of time reading a second output 

30 from each said pixel so as to obtain a second set of 

31 image data having a second dynamic range; and 



5 

1 combining said first and second sets of image 

2 data in order to obtain a resultant set of image data 

3 having a further dynamic range different from said 

4 first and second dynamic ranges. 
5 

6 Preferably, the method further comprises, without 

7 resetting said pixels, after at least a third 

8 predetermined period of time reading at least a third 

9 output from each said pixel so as to obtain a third 

10 set of image data having a third dynamic range; and 

11 combining at least said first, second and third 

12 sets of image data in order to obtain a resultant set 

13 of image data having a further dynamic range 

14 different from said first, second and third dynamic 

15 ranges. 
16 

17 In accordance with a second aspect of the invention, 

18 there is provided a method of operating a solid state 

19 image sensor having an image sensing array comprising 
2 0 a plurality of active pixels, the method comprising: 

21 resetting and immediately reading a preliminary 

22 output from each said pixel; 

23 after a first predetermined period of time, 

24 reading a first output from each said pixel. 
25 

2 6 Preferably, the method further includes the step of 

2 7 determining the difference between said preliminary 

28 and first outputs so as to obtain a set of image data 

y 

2 9 substantially free of noise components represented by 

30 said preliminary outputs. 

31 



1 Preferably, the method in accordance with the first 

2 aspect of the invention is combined with the method 

3 in accordance with the second aspect of the 

4 invention, wherein said preliminary outputs of the 

5 second aspect are read immediately after performing 

6 the resetting step of the first aspect. 
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8 Preferably, the method further includes the step of 

9 determining the difference between said preliminary 
outputs and each of said first, second and any 

11 subsequent outputs so as to obtain a plurality of 

12 said sets of image data each of which is 

13 substantially free of noise components represented by 

14 said preliminary outputs. 



15 
16 



Preferably, in each of the aforementioned aspects of 

17 the invention, the or each said predetermined time 

18 period is selected to be an integer multiple of a 

19 predetermined lighting flicker period. 
20 

21 Preferably, in each of the aforementioned aspects of 

22 the invehtion, said image sensing array remains 

23 continuously exposed to incident light while the 

24 method is performed. 
25 

26 In accordance with a further aspect of the invention, 

27 there is provided a solid state image sensor adapted 
to perform a method in accordance with any one of the 
first to third aspects of the invention. 
In accordance with another aspect of the invention, 

31 there is provided a solid state image sensor system 



28 
29 
30 



adapted to perform a method in accordance with any 
one of the first to third aspects of the invention. 

In accordance with still another aspect of the 
invention, there is provided a camera incorporating a 
solid state image sensor or image sensor system 
adapted to perform a method in accordance with any 
one of the first to third aspects of the invention. 

US-A-5926214 discloses image sensors and methods of 
operation thereof wherein multiple read cycles are 
performed between resets of an active pixel sensor 
array. However, these methods are concerned only 
with noise reduction and require the use of an 
optical shutter to mask the array from incident light 
during an initial reset/read cycle and during 
subsequent read operations between successive 
integration periods. 

Embodiments of the invention will now be described, 
by way of example only, with reference to the 
accompanying drawings, in which: 

Figs. 1(a), 1(b) and 1(c) illustrate, respectively, a 
passive type image sensor pixel, a 3 -transistor 
active image sensor pixel and a 4 -transistor active 
image pixel; and 

Fig. 2 illustrates a portion of one example of a 3- 
transistor active pixel image sensor array. 
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1 Referring now to Fig. 2 of the drawings, a 2x2 pixel 

2 portion of a typical active pixel CMOS image sensor 

3 array 1 is shown. The array 1 comprises a plurality 

4 of rows 3, 5... and columns 4, 6... of active 
photosensitive pixels 10 defining an image sensing 
area. The pixels may be addressed sequentially by 
vertical 12 and horizontal 14 shift registers 
electronically connected to the pixels as shown, or 
by any other suitable pixel addressing scheme such as 
a decoded address scheme. The shift registers 12, 14 
are electronically connected to scanning circuitry 
(not shown) for scanning - i.e. reading - the pixel 

13 outputs to an output 0/P. Any of a variety of 
known types of active pixel may be used for the 
pixels of the array. m the array of Fig. 2, the 
pixels 10 each comprise a photodiode 11 and 
associated transistor circuitry for use in amplifying 
(buffering) the diode outputs and for reading and 
resetting the diodes li, as is well known in the art. 
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In conventional use of a sensor array of this type, 
the pixels would normally be reset and light would 
impinge on the photodiodes 11 for a predetermined 
period (the integration period) , before the pixels " 
are read in order to capture a set of image data from 
the array. The pixels would then be reset prior to 
each integration period for each image to be 
captured. 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 



In accordance with the present invention, multiple 
reads are performed between successive resets, as 
follows . 

Firstly, the pixels are reset, destroying any 
previous pixel signals and forcing the photodiode of 
each pixel to a known reset voltage (Vrt) / and are 
read immediately after being reset (preliminary read 
cycle or Read 0) . It will be understood that, as in 
conventional image sensor operation, lines (rows or 
columns) of pixels are reset and read sequentially, 
so that all of the pixels in one line are reset 
simultaneously and then read simultaneously. 
However, this preliminary read cycle is performed 
immediately after resetting the pixels, rather than 
after a predetermined integration period as in 
conventional image sensors. 

The output from each pixel when read immediately 
after reset is: 



Outo = Vrt + Nrst + Voff + V 



img 



Where Vrt is the reset voltage (described above) , Nrst 
is the reset noise (described above and different on 
each reset occasion) , Voff is a circuit-induced 
voltage offset whose value can be unique to each 
pixel due to local random threshold variations, Vi^g 
is a signal due to any stray light integration which 
may have occurred between reset and Read 0. That is, 
Outo comprises a "dark" signal which contains noise 
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1 comprising the above mentioned components. However, 

2 these noise components remain substantially constant 

3 over short periods of time and until a new reset 

4 occurs . 
5 

6 Following Read 0, without resetting the pixels, light 

7 is integrated during a first predetermined 

8 integration period, Tinti/ producing a signal, Sigi, 

9 within each pixel due to the discharge of pixel 

10 capacitance by way of photo-induced leakage current. 

11 A first read cycle. Read 1, is performed at the end 

12 of Tinti- The output from each pixel when read at the 

13 end of Tinti is: 
14 

15 Outi = Vrt + Nrst + Voff + Vi^g + Sigi. 

16 

17 By calculating the difference between Outo and Outi, 

18 the value of Sigi can be determined free of the noise 

19 components which comprise Outo/ since these 

20 components are constant between Read 1 and Read 2; 

21 i.e. 
22 

23 Outi - Outo = Sigi. 

24 

2 5 The read process described thus far therefore 

26 provides an output signal which is substantially free 

27 of noise. 
28 

29 Preferably, at least one further read cycle. Read 2, 

3 0 is performed after a second predetermined integration 
31 period, Tint2/ again without resetting the pixels. 



resulting in further discharge of the photodiode 
capacitance and producing a further signal, Sig2, 
such that 

Out2 - Outi = Sigs; 

and 

Out2 - Outo = Sig3; 

Where Siga = Sigi + Sig2 and where Sig2 and Siga are 
also substantially free of the noise components 
represented by Outo. 

It can be seen that Sigi, Sig2 and Sigs provide sets 
of image data with different exposure periods so that 
the sensor outputs obtained from the three read 
cycles provide three different representations of the 
same scene taken close together in time and having 
different dynamic ranges. Sigi, corresponding to the 
shortest integration period, will contain most 
information from relatively bright image areas but is 
likely to be underexposed in relatively dim image 
areas, Sig2 will provide an intermediate view and 
Siga will contain most information from relatively 
dim image areas but is likely to be overexposed in 
relatively bright image areas. The three images may 
then be combined in order to obtain a composite image 
having a dynamic range wider than could be obtained 
by means of a single integration period. 
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1 The three images may be combined in any of a variety 

2 of ways. Generally speaking, the signals will be 

3 normalised so that the highest luminance values- from 

4 the first (shortest exposure) image are scaled to the 

5 upper end of a predetermined range of output values, 

6 the lowest luminance values from the third (longest 

7 exposure) image are scaled to the lower end of the 

8 range of output values, and intermediate composite 

9 output values are determined by combining and/or 
10 scaling intermediate values from all three images. 
11 

12 The pixels may be reset following the final read 

13 cycle. It will be understood that the number of read 

14 cycles may vary between resets. The method may be 

15 implemented using a substantially conventional image 

16 sensor, with suitably adapted control software and/or 

17 firmware and/or hardware for controlling the timing 

18 of reset and read cycles, sufficient frame storage 

19 resources to store the multiple sets of data captured 
2 0 in each read cycle, and suitable image processing 

21 software and/or firmware and/or hardware for 

22 combining the image data as required. Obviously, the 

23 greater the number of read cycles the greater the 

24 overhead of frame storage and data processing. 
25 

26 It will be understood that the increased dynamic 

27 range obtained by means of multiple integration 

2 8 periods and read cycles may be usefully employed 

29 independently of the preliminary read cycle (Read 0) 

3 0 which allows the cancellation of noise from the 
31 images. However, it is preferred that these 




13 



operations are combined in order to obtain images 

2 having low noise and wide dynamic range. 
3 

4 All of the images obtained by means of such multiple 

5 read cycles may be free of lighting flicker effects 

6 if the integration periods are each selected to be 

7 integer multiples of any lighting flicker period. 
8 

9 It will also be understood that the methods and image 

10 sensors of the present invention do not require the 

11 use of an optical shutter in order to mask the image 

12 sensor either during the initial reset/read operation 

13 or during subsequent read operations. Read 0 is 

14 performed immediately after reset, with resetting and 

15 reading being performed on a sequential, line-by-line 

16 basis, and subsequent reads are performed in a 

17 similar manner while integration continues, 
18 

19 The methods of the invention may be implemented by 

2 0 means of a suitably adapted image sensor, or image . 

21 sensor system, or camera incorporating an image 

22 sensor or image sensor system. The methods are also 

23 applicable to any type of active pixel architecture, 

24 most suitably of the CMOS type, supporting non- 
25 destructive read operations. Additional examples of 

26 such pixel architectures are illustrated in US-A- 

27 5926214. 
28 

2 9 . Improvements and modifications may be incorporated 

30 without departing from the scope of the invention as 

31 defined in the Claims appended hereto. 
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1 Claims 
2 

3 1. A method of operating a solid state image sensor 

4 having an image sensing array comprising a plurality 

5 of active pixels, the method comprising: 

6 resetting each said pixel; 

7 after a first predetermined period of time 

8 reading a first output from each said pixel so as to 

9 obtain a first set of image data having a first 

1 0 dynami c range ; 

11 without resetting said pixels, after a second 

12 predetermined period of time reading a second output 

13 from each said pixel so as to obtain a second set of 

14 image data having a second dynamic range; and 

15 combining said first and second sets of image 

16 data in order to obtain a resultant set of image data 

17 having a further dynamic range different from said 

18 first and second dynamic ranges. 
19 

20 2 . A method as claimed in Claim 1, further 

21 comprising, without resetting said pixels, after at 

22 least a third predetermined period of time reading at 

23 least a third output from each said pixel so as to 

24 obtain a third set of image data having a third 
2 5 dynamic range; and 

26 combining at least said first, second and third 

27 sets of image data in order to obtain a resultant set 

2 8 of image data having a further dynamic range 

29 different from said first, second and third dynamic 

3 0 ranges. 
31 
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2 3. A method of operating a solid state image sensor 

3 having an image sensing array comprising a plurality 

4 of active pixels, the method comprising: 

5 resetting and immediately reading a preliminary 

6 output from each said pixel; 

7 after a first predetermined period of time, 

8 reading a first output from each said pixel. 
9 

10 4. A method as claimed in Claim 3, further 

11 including the step of determining the difference 

12 between said preliminary and first outputs so as to 

13 obtain a set of image data substantially free of 

14 noise components represented by said preliminary 

15 outputs. 
16 

17 5. A method as claimed in Claim 1 or Claim 2, in 

18 combination with a method as claimed in Claim 3 or 

19 Claim 4, wherein said preliminary outputs of Claim 3 

20 or Claim 4 are read immediately after performing the 

21 resetting step of Claim 1 or Claim 2. 
22 

23 6. A method as claimed in Claim 5, further 

24 including the step of determining the difference 

25 between said preliminary outputs and each of said 

26 first, second and any subsequent outputs so as to 

27 obtain a plurality of said sets of image data each of 

28 which is substantially free of noise components 

29 represented by said preliminary outputs. 
30 
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1 7. A method as claimed in any preceding Claim, 

2 wherein the or each said predetermined time period is 

3 selected to be an integer multiple of a predetermined 

4 lighting flicker period. 
5 

6 8. A method as claimed in any preceding Claim, 

7 wherein said image sensing array remains continuously 

8 exposed to incident light while the method is 

9 performed, 
10 

11 9. A solid state image sensor adapted to perform a 

12 method as claimed in any one of Claims 1 to 8 . 
13 

14 10. A solid state image sensor system adapted to 

15 perform a method as claimed in any one of Claims 1 to 

16 8. 
17 

18 11. A camera incorporating a solid state image 

19 sensor or image sensor system adapted to perform a 

20 method as claimed in any one of Claims 1 to 8 . 
21 

22 
23 



